---
id: testing-distributed-services-with-tyk-opentelemetry-tracetest
title: Testing Distributed Services Proxied by Tyk Gateway (API Gateway) with OpenTelemetry and Tracetest
description: Quickstart on how to configure Tyk Gateway with OpenTelemetry and Tracetest to execute trace-based tests against your Distributed Services.
hide_table_of_contents: false
keywords:
  - tracetest
  - trace-based testing
  - observability
  - distributed tracing
  - testing
  - end to end testing
  - end-to-end testing
  - integration testing
  - tyk
  - api gateway
  - opentelemetry
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
---

:::note
[Check out the source code on GitHub here.](https://github.com/kubeshop/tracetest/tree/main/examples/quick-start-tyk)
:::

[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions.

[Tyk Gateway](https://tyk.io/) is an open-source API gateway and management platform designed to help developers control and manage their APIs. It serves as an intermediary layer between client applications and backend services, providing functionalities like authentication, rate limiting, access control, analytics, and traffic management.

## Why is this important?

Testing Distributed Services behind API Gateways has been a pain point for years. Not having visibility into the infrastructure and not knowing where a test fails causes the MTTR to be higher than for other tools. Including OpenTelemetry in your stack, allows you to expose telemetry from the tools you use like Tyk Gateway and your services that you can use for both production visibility and trace-based testing.

This sample shows how to run integration tests against a Node.js API behind Tyk Gateway, using [OpenTelemetry](https://opentelemetry.io/) and Tracetest.

The Node.js Services will fetch data from an external API, transform the data, and insert it into a Postgres table. This particular flow has two failure points that are difficult to test.

1. Validating that an external API request from the worker function is successful.
2. Validating that the Postgress insert operation is successful.

## Prerequisites

**Tyk Gateway Example:**

Clone the [Tracetest GitHub Repo](https://github.com/kubeshop/tracetest) to your local machine, and open the quick start tyk quick start example app.

```bash title=Terminal
git clone https://github.com/kubeshop/tracetest.git
cd tracetest/examples/quick-start-tyk
```

**Tracetest Account**:

- Sign up to [`app.tracetest.io`](https://app.tracetest.io) or follow the [get started](/getting-started/overview) docs.
- Have access to the environment's [agent API key](https://app.tracetest.io/retrieve-token).

**Docker**: Have [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your machine.

## Project Structure

This is a Docker Compose project you can find the setup in the `docker-compose.yml` file

### 1. The Tyk Gateway

In the `docker-compose.yml` file you can find the Tyk Gateway setup.

```yaml title=docker-compose.yml
# Tyk Gateway
tyk-gateway:
  image: tykio/tyk-gateway:v5.2.1
  ports:
    - 8080:8080
  environment:
    - TYK_GW_OPENTELEMETRY_ENABLED=true
    - TYK_GW_OPENTELEMETRY_EXPORTER=grpc
    - TYK_GW_OPENTELEMETRY_ENDPOINT=otel-collector:4317
  volumes:
    - ./deployments/tyk-gateway/apps:/opt/tyk-gateway/apps
    - ./deployments/tyk-gateway/tyk.conf:/opt/tyk-gateway/tyk.conf
  depends_on:
    - tyk-redis

tyk-redis:
  image: redis:6.0.4
  volumes:
    - tyk-redis-data:/data
```

Adding the configuration for OpenTelemetry to the Tyk Gateway is as simple as setting the environment variables `TYK_GW_OPENTELEMETRY_ENABLED`, `TYK_GW_OPENTELEMETRY_EXPORTER`, and `TYK_GW_OPENTELEMETRY_ENDPOINT`.

You can find the Tyk Gateway apps and configuration in the `deployments/tyk-gateway` file.

```yaml title=deployments/tyk-gateway/apps/pokeshop-demo.json
{
  "name": "pokeshop",
  "api_id": "1",
  "org_id": "default",
  "active": true,
  "use_keyless": false,
  "detailed_tracing": true,
  "definition": { "location": "header", "key": "version" },
  "auth": { "auth_header_name": "authorization" },
  "version_data": { "not_versioned": true, "versions": { "Default": { "name": "Default" } } },
  "proxy":
    { "listen_path": "/", "target_url": "http://api:8081/", "strip_listen_path": true, "preserve_host_header": true },
}
```

### 2. Tracetest

The tracetest setup is composed by the `tracetest-e2e` and the `tracetest-agent` services under the `docker-compose.yml` file.

```yaml title=docker-compose.yml
# Cloud-based Managed Tracetest
tracetest-agent:
  image: kubeshop/tracetest-agent:latest
  environment:
    # Get the required information here: https://app.tracetest.io/retrieve-token
    - TRACETEST_API_KEY=${TRACETEST_TOKEN}
    - TRACETEST_ENVIRONMENT_ID=${TRACETEST_ENVIRONMENT_ID}

tracetest-apply:
  build:
    dockerfile: Dockerfile.tracetest
  volumes:
    - ./resources:/resources
  environment:
    TRACETEST_TOKEN: ${TRACETEST_TOKEN}
    TRACETEST_ENVIRONMENT_ID: ${TRACETEST_ENVIRONMENT_ID}
  entrypoint:
    - bash
    - /resources/apply.sh
  networks:
    default: null
  depends_on:
    api:
      condition: service_started
    tracetest-agent:
      condition: service_started

tracetest-run:
  build:
    dockerfile: Dockerfile.tracetest
  volumes:
    - ./resources:/resources
  environment:
    TRACETEST_TOKEN: ${TRACETEST_TOKEN}
    TRACETEST_ENVIRONMENT_ID: ${TRACETEST_ENVIRONMENT_ID}
  entrypoint:
    - bash
    - /resources/run.sh
  networks:
    default: null
  depends_on:
    tracetest-apply:
      condition: service_completed_successfully
```

### 3. Services under Test

As a testing ground, the example uses Tracetest's own Pokeshop Demo APP which includes the `api` and `worker` services.

```yaml title=docker-compose.yml
# Demo services

# pokeshop demo services
postgres:
  image: postgres:14
  environment:
    POSTGRES_USER: postgres
    POSTGRES_PASSWORD: postgres
    POSTGRES_DB: postgres
  healthcheck:
    test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]
    interval: 1s
    timeout: 5s
    retries: 60

cache:
  image: redis:6
  healthcheck:
    test: ["CMD", "redis-cli", "ping"]
    interval: 1s
    timeout: 3s
    retries: 60

queue:
  image: rabbitmq:3.12
  restart: unless-stopped
  healthcheck:
    test: rabbitmq-diagnostics -q check_running
    interval: 1s
    timeout: 5s
    retries: 60

api:
  image: kubeshop/demo-pokemon-api:latest
  restart: unless-stopped
  pull_policy: always
  environment:
    REDIS_URL: cache
    DATABASE_URL: postgresql://postgres:postgres@postgres:5432/postgres?schema=public
    RABBITMQ_HOST: queue
    POKE_API_BASE_URL: https://pokeapi.co/api/v2
    COLLECTOR_ENDPOINT: http://otel-collector:4317
    NPM_RUN_COMMAND: api
  healthcheck:
    test: ["CMD", "wget", "--spider", "localhost:8081"]
    interval: 1s
    timeout: 3s
    retries: 60
  depends_on:
    postgres:
      condition: service_healthy
    cache:
      condition: service_healthy
    queue:
      condition: service_healthy

worker:
  image: kubeshop/demo-pokemon-api:latest
  restart: unless-stopped
  pull_policy: always
  environment:
    REDIS_URL: cache
    DATABASE_URL: postgresql://postgres:postgres@postgres:5432/postgres?schema=public
    RABBITMQ_HOST: queue
    POKE_API_BASE_URL: https://pokeapi.co/api/v2
    COLLECTOR_ENDPOINT: http://otel-collector:4317
    NPM_RUN_COMMAND: worker
  depends_on:
    postgres:
      condition: service_healthy
    cache:
      condition: service_healthy
    queue:
      condition: service_healthy
```

## Set up Environment Variables

Copy the `.env.template` and create a `.env` file in the same directory. Add token and Cloud Agent endpoint.

```txt title=.env
# Get the required information here: https://app.tracetest.io/retrieve-token

TRACETEST_TOKEN="<YOUR_TRACETEST_TOKEN>"
TRACETEST_ENVIRONMENT_ID="<YOUR_ENV_ID>"

POKESHOP_DEMO_URL=http://tyk-gateway:8080
TYK_AUTH_KEY=28d220fd77974a4facfb07dc1e49c2aa
```

## The Tracetest End To End Script

The `resources/script.js` file contains the script that will execute the trace-based tests against the Pokeshop App proxied by a Tyk endpoint.

### Steps Executed by the Script

1. Create a new key in the Tyk Gateway.
3. Import a Pokemon.

```typescript title=resources/script.js
const { expect } = require('@playwright/test');

const URL = 'http://tyk-gateway:8080';
const API_KEY = '28d220fd77974a4facfb07dc1e49c2aa';

const getKey = async () => {
  const params = {
    headers: {
      'Content-Type': 'application/json',
      'x-tyk-authorization': API_KEY,
      'Response-Type': 'application/json',
    },
  };

  const data = {
    alias: 'website',
    expires: -1,
    access_rights: {
      1: {
        api_id: '1',
        api_name: 'pokeshop',
        versions: ['Default'],
      },
    },
  };

  const res = await fetch(`${URL}/tyk/keys/create`, {
    ...params,
    method: 'POST',
    body: JSON.stringify(data),
  });

  const { key } = await res.json();

  return key;
};

async function importPokemon(page) {
  const key = await getKey();

  await page.setExtraHTTPHeaders({
    Authorization: `Bearer ${key}`,
  });

  await page.goto(URL);

  expect(await page.getByText('Pokeshop')).toBeTruthy();

  await page.click('text=Import');
  await page.getByLabel('ID').fill('143');

  await Promise.all([
    page.waitForResponse((resp) => resp.url().includes('/pokemon/import') && resp.status() === 200),
    page.getByRole('button', { name: 'OK', exact: true }).click(),
  ]);
}

module.exports = { importPokemon };
```

The output from the Playwright tests will show the test results with links to the Tracetest App.

```bash title=Terminal
[+] Running 2/2
 ✔ worker Pulled                                                                                                                                                                     0.9s
 ✔ api Pulled                                                                                                                                                                        0.8s
[+] Creating 8/0
 ✔ Container quick-start-tyk-tracetest-agent-1  Running                                                                                                                              0.0s
 ✔ Container quick-start-tyk-postgres-1         Running                                                                                                                              0.0s
 ✔ Container quick-start-tyk-jaeger-1           Running                                                                                                                              0.0s
 ✔ Container quick-start-tyk-tyk-redis-1        Running                                                                                                                              0.0s
 ✔ Container quick-start-tyk-cache-1            Running                                                                                                                              0.0s
 ✔ Container quick-start-tyk-otel-collector-1   Running                                                                                                                              0.0s
 ✔ Container quick-start-tyk-queue-1            Running                                                                                                                              0.0s
 ✔ Container quick-start-tyk-tyk-gateway-1      Running                                                                                                                              0.0s
[+] Running 7/7
 ✔ Container quick-start-tyk-jaeger-1           Healthy                                                                                                                              0.5s
 ✔ Container quick-start-tyk-postgres-1         Healthy                                                                                                                              3.2s
 ✔ Container quick-start-tyk-cache-1            Healthy                                                                                                                              3.2s
 ✔ Container quick-start-tyk-queue-1            Healthy                                                                                                                              3.2s
 ✔ Container quick-start-tyk-worker-1           Started                                                                                                                              0.2s
 ✔ Container quick-start-tyk-api-1              Started                                                                                                                              0.1s
 ✔ Container quick-start-tyk-tracetest-apply-1  Started                                                                                                                              0.2s
[+] Running 2/2
 ✔ api Pulled                                                                                                                                                                        0.8s
 ✔ worker Pulled                                                                                                                                                                     0.8s
Configuring Tracetest
 SUCCESS  Successfully configured Tracetest CLI
Running Trace-Based Tests...
✔ RunGroup: #QMjWZ5jIR (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/run/QMjWZ5jIR)
 Summary: 1 passed, 0 failed, 0 pending
  ✔ Import Pokemon (https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/4_IKfPlIg/run/13/test) - trace id: 1554d5d5904daa17f2a6854a7d7608e5
	✔ Import should return 200
	✔ All Database Spans: Processing time is less than 1s
	✔ Import Pokemon Span Should be Present
	✔ Redis should return in 20ms
```

### Tracetest App Results

![Tracetest App Results](../img/tyk-results.png)

### Running the Example

Spin up the deployment and test execution.

```bash title=Terminal
docker-compose run tracetest-run
```

This will trigger the Docker Compose setup and immediately run the [trace-based tests using the Tracetest Playwright integration](../../tools-and-integrations/playwright.mdx) as part of the `tracetest-e2e` service.

## Learn More

Feel free to check out our [examples in GitHub](https://github.com/kubeshop/tracetest/tree/main/examples) and join our [Slack Community](https://dub.sh/tracetest-community) for more info!
